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ABSTRACT 

Detecting  mud  hazards  is  a  significant  challenge  to 
UGV  autonomous  off-road  navigation.  A  military  UGV 
stuck  in  a  mud  body  during  a  mission  may  need  to  be 
sacrificed  or  rescued,  both  unattractive  options.  JPL  is 
currently  developing  a  daytime  mud  detection  capability 
under  the  RCTA  program  using  UGV  mounted  sensors. 
To  perform  robust  mud  detection  under  all  conditions,  we 
expect  multiple  sensors  will  be  necessary.  A  passive  mud 
detection  solution  is  desirable  to  meet  the  FCS-ANS 
requirements.  To  characterize  the  advantages  and 
disadvantages  of  candidate  passive  sensors,  data 
collections  have  been  performed  on  wet  and  dry  soil  using 
visible,  multi- spectral  (including  near- infrared),  short¬ 
wave  infrared,  mid-wave  infrared,  long-wave  infrared, 
polarization,  and  stereo  sensors.  In  this  paper,  we 
examine  the  cues  for  mud  detection  each  of  these  sensors 
provide,  along  with  their  deficiencies,  and  we  illustrate 
localizing  detected  mud  in  a  world  model  that  can  used  by 
a  UGV  to  plan  safe  paths. 

1.  INTRODUCTION 

There  are  many  types  of  terrain  (such  as,  pavement, 
gravel,  firm  dirt,  mud,  snow,  ice,  water,  and  vegetation  of 
varying  height  and  thickness)  that  an  unmanned  ground 
vehicle  (UGV)  can  encounter  during  autonomous 
navigation  over  cross  country  terrain.  The  ability  to 
detect  and  avoid  hazardous  terrain  poses  a  current 
challenge  to  the  acceptance  and  proliferation  of  tactical 
UGVs  (Mills,  2007).  Since  water  and  mud  are  prevalent 
in  many  regions  during  the  entire  year,  robust  detection  of 
those  hazards  is  a  critical  perception  requirement  for 
UGV  autonomous  navigation. 

Water  and  mud  bodies  need  to  be  detected  at  a 
sufficient  distance  (which  is  a  function  of  the  UGV’s 
current  speed)  that  allows  the  UGV  to  evade  it.  A  UGV 
that  becomes  stuck  in  water  or  mud  during  an  in-theater 
autonomous  mission  may  require  rescue,  potentially 
drawing  critical  resources  away  from  the  primary  mission 
and  soldiers  into  harms  way.  Under  the  Army  Research 
Laboratory  (ARL)  Robotics  Collaborative  Technology 
Alliances  (RCTA)  program,  the  Jet  Propulsion  Laboratory 
(JPL)  is  developing  a  water  and  mud  detection  capability 
for  UGVs.  At  the  last  Army  Science  Conference,  we 
reported  our  progress  on  detecting  and  localizing  water 


bodies  in  a  world  model  used  to  plan  safe  paths  (Rankin 
and  Matthies,  2006).  This  year,  we  report  our  progress  on 
evaluating  sensors  applicable  to  mud  detection  from  an 
UGV. 

Figure  1  illustrates  the  class  of  UGV  this  work  is 
being  performed  for  and  a  severe  mud  hazard  from  a  test 
site  near  General  Dynamics  Robotic  Systems, 
Westminster,  MD.  Although  mud  detection  for  UGV 
autonomous  navigation  is  a  relatively  new  research  area, 
techniques  to  estimate  soil  moisture  content  have  been 
studied  for  decades  for  agricultural  applications  (Hamrita 
et  al.,  2000).  Soil  moisture  content  has  been  estimated 
using  devices  requiring  contact  with  the  ground  and  via 
remote  sensing  (Whalley  and  Stafford,  1992;  Suddeth  et 
ah,  1997). 


Figure  1.  An  experimental  unmanned  vehicle  (XUV) 
navigating  through  a  muddy  grass  field  near  General 
Dynamics  Robotic  Systems  (Westminster,  MD). 


Remote  sensing  of  soil  moisture  has  been  performed 
with  sensors  mounted  to  satellites,  aerial  vehicles,  and 
ground  vehicles.  Satellite  methods  of  soil  moisture 
estimation  use  thermal  infrared  (Wetzel  et  ah,  1984), 
passive  and  active  microwave  (Wigneron  et  ah,  1998; 
Dubois  and  van  Zyl  Engman,  1995),  and  multi-spectral 
sensors  (Zeng  et  ah,  2004).  The  satellite  methods  are 
useful  in  characterizing  the  soil  moisture  content  of  large 
fields,  but  lack  the  resolution  necessary  for  detecting 
small  but  severe  UGV  mud  hazards.  Aerial  methods  of 
soil  moisture  estimation  use  passive  and  active 
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microwave  sensors  (O’Neill  et  al.,  1996;  Archer  et  al., 
2004).  Unmanned  aerial  vehicles  (UAVs)  with 
microwave  sensors  have  been  used  to  estimate  soil 
moisture  through  vegetation  clutter  (such  as  a  com  field). 
However,  they  also  lack  the  resolution  for  detecting  small 
but  severe  UGV  mud  hazards. 

Ground  vehicle  methods  of  soil  moisture  estimation 
have  also  used  passive  microwave  sensors  (Archer  et  al., 
2006).  But  the  antennas  tend  to  be  bulky  and  have  been 
mounted  directly  downwards,  requiring  a  UGV  to  drive 
on  potentially  hazardous  terrain  to  characterize  it.  Dupont 
et  al.  (2008)  classified  terrain  types,  included  mud,  by 
using  proprioceptive  sensors  that  measure  UGV  vibration 
and  estimate  applied  wheel  torques.  Angelova  et  al. 
(2007)  developed  an  algorithm  to  predict  the  amount  of 
wheel  slippage  terrain  in  the  distance  will  cause  using 
stereo  vision  sensors.  However,  these  methods  also 
require  a  UGV  to  drive  on  potentially  hazardous  terrain  to 
characterize  or  learn  about  it. 

Alshikaili  (2007)  used  a  non-contact  infrared 
thermocouple  to  predict  soil  moisture  content  based  on 
soil  temperature  from  a  tripod.  But  a  fieldable  weather 
station  was  required  to  determine  some  model  parameters 
(air  temperature,  solar  radiation,  relative  humidity,  wind 
speed),  soil  emissivity  needed  to  be  manually  selected  on 
the  thermocouple,  and  the  level  of  soil  compaction  needed 
to  be  specified.  Powers  (2008)  is  experimenting  with  a 
monostatic  active  polarimeter  designed  to  classify  solid 
ground  and  mud  for  UGVs.  But  because  there  are 
military  operations  when  it  may  be  desirable  for  UGVs  to 
operate  without  emitting  strong,  detectable  electro¬ 
magnetic  signals,  a  passive  perception  solution  to  mud 
detection  is  also  desirable. 

Under  the  RCTA  program,  JPL  has  performed 
daytime  data  collections  on  wet  and  dry  soil  using  a 
variety  of  passive  imaging  sensors:  color  stereo,  multi- 
spectral  including  near-infrared  (NIR),  short-wave 
infrared  (SWIR),  mid- wave  infrared  (MWIR),  long- wave 
infrared  (LWIR),  and  polarization.  In  this  paper,  we 
examine  the  cues  for  mud  detection  each  of  these  sensors 
can  provide,  along  with  their  deficiencies.  We  started  this 
work  focusing  primarily  on  detecting  mud  where  it  is 
most  likely  to  be  encountered  by  a  UGV :  on  dirt  trials  or 
open  dirt  fields.  The  example  in  Figure  1  illustrates 
probably  the  hardest  UGV  mud  perception  scenario; 
detecting  mud  when  the  entire  ground  is  wet  and  the 
surface  is  not  soil.  Difficult  scenarios  such  as  these  will 
be  addressed  in  future  work.  Here,  we  mostly  limit  our 
examination  to  mud  detection  during  the  daytime  under 
ideal  conditions:  isolated  wet  soil  surrounded  by  dry  soil 
during  nominal  weather,  i.e.,  no  precipitation,  calm  wind, 
and  near  average  temperatures. 


2.  MUD  CUES  FROM  COLOR 

It  is  commonly  observed  that  wet  soil  is  considerably 
darker  than  dry  soil  in  visible  imagery.  This  is  because 
changing  the  medium  surrounding  soil  particles  from  air 
to  water  decreases  the  particle’s  relative  refractive  index, 
increasing  the  average  degree  of  forwardness  of 
scattering.  As  a  result,  incident  photons  have  a  higher 
probability  of  being  absorbed  (Twomey  et  al.,  1986). 
Perrson  (2005)  found  that  soil  gradually  becomes  darker 
as  it  is  wet  to  a  soil  moisture  level  of  0.25m3 m"3  (volume 
water  per  unit  volume  of  soil).  Beyond  that  level,  some 
soils  start  to  become  lighter  again  as  water  starts 
becoming  visible  on  the  surface,  causing  reflections.  As 
seen  in  Figure  1,  small  patches  of  sitting  water  tend  to  be 
visible  when  the  ground  is  saturated.  These  small  patches 
of  water  are  a  cue  for  muddy  terrain  and  can  be  detected 
from  color  analysis  if  the  water  is  reflecting  the  sky 
(Rankin  and  Matthies,  2006). 

The  example  scenes  in  Figure  2  illustrate  that  ground 
regions  significantly  darker  than  surrounding  terrain  is 
also  a  cue  for  mud.  But  shadows  are  also  ground  regions 
significantly  darker  than  surrounding  terrain.  To 
disambiguate  mud  from  shadows,  one  can  analyze  the 
color  content  of  the  dark  region.  Shadows  inherently  do 
not  receive  direct  light  from  the  primary  light  source.  As 
a  result,  blue  light  tends  to  scatter  more  relative  to  red  and 
green  light,  giving  shadows  a  higher  blue  saturation  than 
neighboring  regions  (Wells,  2007).  This  is  illustrated  in 
Figure  3.  In  the  left  hue  image,  the  soil  in  shadow  is  bluer 
than  non-shadowed  soil.  In  the  right  image,  the  non- 
shadowed  mud  has  the  same  hue  as  the  surrounding  dry 
soil. 

In  summary,  isolated  mud  can  be  detected  from  color 
imagery  by  segmenting  regions  significantly  darker  than 
surrounding  terrain  (but  with  the  same  hue  as  the 
surrounding  terrain).  Extremely  muddy  terrain  can  be 
detected  by  segmenting  small  water  patches  that  reflect 
the  sky.  Color  can  also  be  useful  in  disambiguating  non- 
shadowed  mud  from  shadows.  However,  color  is 
probably  not  very  useful  in  determining  if  the  terrain  in  a 
shadow  is  mud,  or  in  distinguishing  the  muddiest  portion 
of  terrain  when  the  entire  ground  is  wet  (except  where 
there  are  small  patches  of  water  reflecting  the  sky). 


Figure  2.  Mud  tends  to  be  darker  than  surrounding 
dry  soil  in  color  imagery. 
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Figure  3.  Hue  images  for  the  color  images  in  Figure  2. 
Soil  in  shadow  tends  to  be  bluer  than  non-shadowed 
wet  or  dry  soil.  But  non-shadowed  wet  soil  tends  to 
have  the  same  hue  as  dry  soil. 

3.  MUD  CUES  FROM  MULTI-SPECTRAL 


vegetation,  and  the  second  looking  down  a  long  dirt  road 
lined  with  vegetation.  Water  was  poured  over  a  small 
portion  of  both  scenes  and  allowed  to  soak  in  to  the 
ground.  Figure  4  contains  images  of  the  red  and  NIR 
bands  for  the  first  scene.  It  also  contains  a  NIR  vs.  red 
reflectance  plot  of  the  pixels  in  the  boxes  overlaid  on  the 
NIR  image.  The  red  box  contains  dry  soil,  the  green  box 
contains  wet  soil,  and  the  blue  box  contains  dry  soil  in 
shadow.  The  yellow  line  is  a  least  squares  line  fit  of  the 
data.  From  this  example,  it  appears  that  the  soil  line 
phenomenology  is  also  valid  from  a  ground-based  sensor. 
Since  dry  soil  in  shadow  plots  to  the  left  of  wet  soil  on  the 
soil  line ,  it  is  more  difficult  to  disambiguate  shadows 
from  wet  soil  than  shadows  from  dry  soil. 


In  the  remote  sensing  community,  it  is  well  known 
that  LandSat  multi-spectral  data  on  bare  soil  falls  on  a  line 
(referred  to  as  the  soil  line )  when  plotted  in  NIR  vs.  red 
reflectance  space.  Bare  soil  pixel  placement  on  the  soil 
line  depends  on  soil  moisture  content.  Wet  bare  soil  maps 
to  one  end  of  the  line  and  dry  bare  soil  maps  to  the 
opposite  end.  To  determine  if  this  phenomenology  is 
valid  from  a  ground-based  sensor,  we  performed  data 
collections  on  wet  and  dry  soil  with  a  Duncan  Tech 
MS2100  multi-spectral  camera,  having  spectral  sensitivity 
from  400-1000nm.  This  camera  is  based  on  a  color 
separating  prism  and  three  imaging  channels  that  allow 
simultaneous  image  acquisition  of  four  spectral  bands 
(red,  green,  blue,  and  NIR)  through  a  common  aperture. 
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Figure  4.  In  NIR  vs.  Red  reflectance  space,  dry,  wet, 
and  shadowed  soil  lies  on  a  line  referred  to  as  the  soil 
line  in  remote  sensing  literature. 


(g)  Segmented  soil  (h)  Segmented  wet  soil 

Figure  5.  NIR  and  red  bands  of  a  multi-spectral 
camera  may  be  useful  in  separating  vegetation  from 
soil,  and  dry  soil  from  wet  soil. 


Figure  5  illustrates  using  multi- spectral  bands  to 
segment  wet  soil.  Red  and  NIR  bands  (Figures  5a  and  5b) 
can  be  used  to  generate  a  Normalized  Difference 
Vegetation  Index  (NDVI)  image  (Figure  5  c).  If  the  slope 


Two  data  collections  were  performed  with  the  multi- 
spectral  camera  mounted  to  a  tripod:  one  in  an  area  that 
contained  bare  soil  (both  in  and  out  of  shadow)  and 
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and  y-intercept  of  the  soil  line  is  known,  a  normal 
distance  to  the  soil  line  image  (Figure  5e)  and  a  distance 
along  the  soil  line  image  (Figure  5f)  can  be  generated. 
The  NDVI  and  normal  distance  to  the  soil  line  images  can 
be  used  to  segment  soil  from  vegetation  (Figure  5g).  A 
brightness  image  (Figure  5d),  which  is  derived  from  the 
visible  bands,  and  the  distance  along  the  soil  line  image 
can  be  used  to  segment  wet  soil  from  dry  soil  (Figure  5h). 

Water,  snow,  and  ice  all  have  larger  red  reflectance 
values  than  NIR,  bare  soil  has  a  slightly  larger  NIR 
reflectance  than  red,  and  vegetation  has  a  much  larger 
NIR  reflectance  than  red.  Some  typical  NDVI  values 
(from  overhead  sensors)  are  0.7  for  dense  vegetation,  0- 
0.1  for  bare  soil,  -0.046  for  snow  and  ice,  and  -0.257  for 
water.  (Holben,  1986).  NDVI  is  more  useful  than 
distance  to  the  soil  line  in  segmenting  bare  soil  from 
vegetation,  but  distance  along  the  soil  line  is  more  useful 
than  NDVI  in  segmenting  wet  soil  from  dry  soil. 
However,  neither  technique  is  likely  to  be  effective  in 
segmenting  snow  and  ice  from  mud.  Distance  along  the 
soil  line  may  be  useful  in  segmenting  wet  soil  from  soil  in 
shadow,  but  it  is  probably  not  very  useful  in  determining 
if  the  soil  in  shadow  is  wet  or  dry. 

4.  MUD  CUES  FROM  SWIR 

Water  strongly  absorbs  light  in  the  NIR  and  SWIR 
wavelengths,  suggesting  that  wet  soil  may  also.  Lobell 
and  Asner  (2002)  acquired  measurements  of  reflected 
shortwave  radiation  (400-2 5 OOnm)  of  four  different  soils 
at  varying  moisture  levels  in  a  laboratory  setting  using  a 
spectrometer  and  a  calibrated  light  source.  They  report 
that  while  reflectance  for  visible  wavelength  saturates  at  a 
soil  moisture  content  of  20%,  reflectance  for  SWIR 
saturates  at  a  much  higher  soil  moisture  content  (50%). 
They  concluded  that  the  SWIR  region  is  more  suitable  for 
measuring  soil  moisture  content  than  NIR.  The  plot  in 
Figure  6  (from  Mouazen  et  al.,  2006)  illustrates  that 
increasing  the  soil  moisture  content  causes  the  largest 
change  in  reflectance  in  a  narrow  SWIR  band  near  a 
wavelength  of  1450nm.  This  suggests  that  passive  SWIR 
should  be  useful  in  detecting  mud. 

We  performed  data  collections  on  wet  and  dry  soil 
with  a  Sensors  Unlimited  SU320M-1.7RT  SWIR  camera, 
to  test  its  feasibility  for  detecting  mud.  This  camera 
contains  a  320x240  InGaAs  detector,  spectrally  sensitive 
from  900-1700nm.  After  acquiring  imagery  of  dry  soil,  a 
few  buckets  of  water  were  poured  on  the  soil.  Sample 
visible  and  SWIR  images  from  the  data  collection  are 
shown  in  Figure  7.  As  expected,  the  wet  soil  is  darker 
than  the  dry  soil  in  visible  and  SWIR  imagery.  We  have 
not  yet  performed  this  experiment  with  varying  soil 
moisture  levels.  We  would  expect  the  wet  soil  to  become 
progressively  darker  as  the  soil  moisture  content 
increases. 


Wavelength,  nm 

Figure  6.  Average  spectra  recorded  for  46  soil  samples 
from  a  single  field  (from  Mouazen  et  al.,  2006).  The 
curves  correspond  to  soil  moisture  content,  having 
units  of  xO.Ol  kg  water  per  1  kg  of  soil. 


Figure  7.  Visible  and  SWIR  imagery  of  dry  and  wet 
soil,  acquired  from  a  UGV.  The  SWIR  sensor  has  a 
narrower  field  of  view  than  the  visible  sensor. 


However,  not  all  the  dark  regions  in  SWIR  imagery 
are  candidate  mud.  Snow,  ice,  and  water  all  cause  dark 
regions  in  SWIR  imagery  (Matthies  et  al.,  2003).  Snow 
and  ice  can  be  distinguished  from  mud  since  they  are 
bright  in  color  imagery  and  mud  is  not.  Water  can  be 
distinguished  from  mud  by  using  color  to  detect  sky 
reflections  and  stereo  range  data  to  detect  terrain 
reflections  (Rankin  and  Matthies,  2006).  Under 
conditions  where  the  entire  ground  is  wet,  if  the  soil 
moisture  content  varies  over  the  terrain,  SWIR  may  be 
able  indicate  the  muddiest  terrain. 

5.  MUD  CUES  FROM  THERMAL  INFRARED 

Factors  that  influence  soil  temperature  include  the 
angle  to  the  sun,  the  soil  moisture  content,  if  the  soil  is  in 
shadow,  and  the  time  of  day.  Soil  color  also  influences 
temperature  since  darker  colors  tend  to  absorb  and  release 
energy  more  rapidly  than  lighter  colors.  Wet  soil  tends  to 
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be  cooler  than  dry  soil  for  a  couple  of  reasons.  First, 
evaporation  helps  to  cool  moist  soil  by  dissipating  heat. 
Second,  the  high  heat  capacity  of  water  reduces  the 
temperature  change  from  the  heat  that  is  absorbed  by  the 
soil.  (Troeh  et  al.,  2005) 

To  test  the  feasibility  of  using  a  thermal  infrared 
imaging  sensor  for  mud  detection,  we  performed  an  all 
day  data  collection  (6:20am-7:30pm)  on  a  mud  body  with 
a  Thermoteknix  Systems  Miricle  110KB  long-wave 
infrared  (LWIR)  camera,  recording  images  every  20 
minutes.  This  camera  has  a  384x288  Alpha  Silicon 
micro-bolometer  detector  with  a  spectral  sensitivity  from 
7- 14 jam.  On  the  day  of  this  data  collection,  there  was  a 
thick  marine  layer  until  9:20am,  it  was  partially  overcast 
until  about  10:00am,  and  then  it  was  sunny  for  the 
remainder  of  the  day. 


Figure  8.  Under  nominal  weather  conditions,  mud  has 
a  strong  signature  in  LWIR  imagery  throughout  the 
entire  day.  AT  is  the  difference  in  temperature 
between  the  mud  and  the  surrounding  dry  soil.  This  is 
the  same  mud  body  in  the  right  image  of  Figure  2. 


during  the  entire  day,  even  when  there  was  heavy 
overcast.  The  temperature  difference  between  the  dry  soil 
and  mud  started  out  low  in  the  morning,  rose  to  a  peak  in 
the  mid  afternoon,  and  started  to  decline  through  the 
evening.  The  sample  imagery  confirms  that  under 
nominal  weather  conditions,  wet  soil  is  cooler  than  dry 
soil  throughout  the  day  in  thermal  infrared  imagery.  Even 
at  the  lowest  temperature  difference  (1.5°C),  there  was 
still  significant  thermal  contrast  between  the  dry  soil  and 
the  mud  in  the  LWIR  imagery. 

However,  not  all  the  dark  regions  in  thermal  infrared 
imagery  are  candidate  mud.  Shadows,  snow,  ice, 
vegetation,  and  water  can  all  cause  dark  regions  in 
thermal  infrared  imagery.  Snow  and  ice  can  be 
distinguished  from  mud  since  they  are  bright  in  color 
imagery  and  mud  is  not.  Tall  vegetation  can  be 
distinguished  from  mud  by  using  stereo  range  data  to 
separate  ground  clutter  from  the  ground  surface.  Water 
can  be  distinguished  from  mud  by  using  color  to  detect 
sky  reflections  and  stereo  range  data  to  detect  terrain 
reflections  (Rankin  and  Matthies,  2006).  Figure  9 
illustrates  that  under  nominal  weather  conditions,  mud  in 
shadow  is  cooler  than  mud  out  of  shadow  during  the 
daytime.  As  discussed  in  section  2,  shadows  can  be 
distinguished  from  non-shadowed  mud  by  analyzing  the 
hue  component  of  color. 


Figure  9.  Under  nominal  weather  conditions,  mud  is 
cooler  than  dry  soil,  and  shadowed  mud  is  cooler  than 
non-shadowed  mud  during  the  daytime. 


Figure  10.  When  the  entire  ground  is  wet  after 
raining,  it  would  be  difficult  to  use  thermal  infrared 
imagery  to  detect  the  muddiest  portions  of  the  terrain. 
This  color  and  MWIR  image  is  from  Ft.  Polk,  LA. 


Figure  8  shows  a  plot  of  the  temperature  of  the  dry 
soil,  mud,  and  air  over  the  entire  day,  and  sample  LWIR 
images  from  6:20am,  10:00am,  3:00pm,  and  7:00pm.  In 
the  temperature  plot,  the  mud  was  cooler  than  the  soil 


Thermal  infrared  imagery  can  provide  a  strong  cue 
for  mud  during  nominal  weather  conditions  when  non¬ 
mud  dark  regions  are  disambiguated  with  other  sensors. 
But  under  off-nominal  conditions,  such  as  after 
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precipitation  when  the  entire  ground  is  damp,  the 
usefulness  of  thermal  infrared  imagery  for  mud  detection 
decreases.  As  illustrated  with  the  mid- wave  infrared 
(MWIR)  image  in  Figure  10,  it  is  difficult  to  detect  the 
muddiest  portions  of  the  road  from  the  rest  of  the  wet 
road  in  thermal  infrared  imagery.  If  the  soil  moisture 
content  varies  over  the  terrain,  SWIR  may  help  in  such 
situations. 

6.  MUD  CUES  FROM  POLARIZATION 

There  are  four  fundamental  properties  of  light: 
intensity,  wavelength,  coherence,  and  polarization.  In  the 
past,  vision-based  terrain  classification  methods  have 
focused  primarily  on  intensity  and  wavelength  (color) 
properties.  But  with  the  recent  emergence  of  polarization 
cameras,  the  components  of  polarization  from  light 
reflected  off  surfaces  can  also  be  exploited  to  classify 
terrain  types.  For  example,  Xie  et  al.  (2007)  and  Pandian 
(2008)  have  recently  experimented  with  using  a 
polarization  camera  to  detect  water. 

Almost  all  naturally  occurring  outdoor  light 
(scattered  and  reflected)  is  at  least  partially  linearly 
polarized  (Wolff,  2005).  At  a  pixel  level,  partial  linear 
polarization  is  measured  by  the  transmitted  radiance 
through  a  polarization  filter.  To  determine  the  whole 
partial  linear  polarization  state  [orientation,  intensity,  and 
degree  of  linear  polarization  (DOLP)],  three  images  of  a 
scene  needs  to  be  acquired,  each  with  the  polarization 
filter  at  a  different  orientation.  The  DOLP  at  each  pixel 
is  a  value  between  0  (completely  unpolarized  light)  and  1 
(completely  linear  polarized  light). 

To  calculate  a  simplified  polarization  measure  called 
polarization  contrast ,  only  two  images  of  a  scene  are 
required,  as  long  as  the  polarization  filter  orientations 
differ  by  90°.  If  one  of  the  filter  orientations  is  aligned 
with  the  orientation  of  the  linearly  polarized  component  in 
the  input  light,  then  polarization  contrast  equals  DOLP. 
Otherwise,  polarization  contrast  provides  a  lower  bound 
for  DOLP  (Pandian,  2008).  Since  water  and  mud  appear 
to  be  horizontally  polarized,  if  filter  orientation  angles  of 
0°  ( crossed)  and  90°  (parallel)  are  chosen,  polarization 
contrast  can  be  used  to  estimate  DOLP  on  water  and  mud. 
The  polarization  contrast  at  each  pixel  can  be  calculated 
by  dividing  the  absolute  value  of  the  difference  between 
the  parallel  and  crossed  intensity  values  by  the  sum  of  the 
parallel  and  crossed  intensity  values. 

Regions  of  a  natural  terrain  scene  that  have  a 
significantly  higher  DOLP  than  surrounding  terrain  can 
be  a  potential  cue  for  water.  However,  the  DOLP  of 
reflections  from  water  bodies  is  a  function  of  the 
incidence  angle,  hence  lookahead  distance.  Moreover,  it 
will  also  be  affected  by  the  polarization  of  the  light 
incident  on  the  water,  which  is  dependent  on  sky 


conditions,  sun  position,  and  viewing  geometry,  as  well  as 
on  the  polarization  of  light  upwelling  from  within  the 
water  body.  Therefore,  there  are  conditions  where  the 
DOLP  on  water  will  not  be  significantly  higher  than 
surrounding  terrain.  As  an  example,  Xie  et  al.  (2007) 
found  that  water  reflecting  nearby  vegetation  had  a  DOLP 
almost  the  same  as  the  surrounding  terrain. 

To  determine  if  there  is  a  polarization  signature  on 
wet  soil  useful  to  detecting  mud,  several  daytime  data 
collections  were  performed  with  two  Bossa  Nova 
Technologies  sensors:  a  SAMBA  polarization  camera  and 
a  SALSA  linear  stokes  polarization  camera.  The 
SAMBA  camera  provides  a  polarization  contrast  image 
and  the  SALSA  camera  provides  DOLP,  intensity,  and 
orientation  images.  In  the  first  data  collection,  SAMBA 
polarization  imagery  of  a  single  mud  body  was  acquired 
every  20  minutes  for  over  seven  hours  from  a  stationary 
position  with  the  sensor  facing  northeast.  The  data 
collection  started  at  6:20am  and  ended  at  1:40pm.  A  full 
day  data  collection  was  planned,  but  a  portable  generator 
that  provided  power  to  the  hardware  failed  near  the  half 
way  point. 


Figure  11.  Segmenting  mud  using  SAMBA 
polarization  imagery  at  different  times  of  the  day. 

A  circular  region  of  soil  having  a  radius  of 
approximately  2  meters  was  periodically  watered  for  three 
days  prior  to  the  data  collection  to  ensure  a  mud 
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consistency.  There  was  a  thick  marine  layer  at  the 
beginning  of  the  data  collection,  but  by  9:20am,  the  sun 
started  to  peek  through  it,  and  by  10:00am,  it  had 
completely  evaporated.  The  polarization  contrast  on  the 
mud  was  significantly  higher  than  the  surrounding  dry 
soil  in  all  the  samples.  Figure  1 1  shows  the  parallel  and 
crossed  polarization  images,  the  resultant  polarization 
contrast  images,  and  segmented  mud  images  for  the  first 
and  last  images  from  the  data  collection.  The  same 
parameters  were  used  to  segment  the  mud  at  both  times. 

In  a  second  data  collection,  a  color  camera  and 
SALSA  polarization  camera  were  mounted  to  the  hood  of 
a  vehicle  and  a  mud  body  was  imaged  from  three  different 
directions  (facing  West,  South,  and  East).  This  data 
collection  took  place  at  approximately  10:00am  on  an 
overcast  day.  Each  image  was  taken  minutes  apart  from 
the  previous  image.  As  illustrated  in  Figure  12,  in  all 
three  cases,  the  DOLP  on  the  mud  was  consistently  higher 
that  the  surrounding  dry  soil. 


Figure  12.  A  mud  body  (A)  imaged  on  an  overcast  day 
from  three  directions  with  a  color  and  SALSA 
polarization  camera.  The  images  for  each  heading 
were  acquired  a  few  minutes  apart.  The  DOLP  on 
mud  is  consistently  high,  independent  of  sensor 
orientation. 


The  DOLP  on  water  can  change  based  on  the  sky 
conditions,  the  sun  position,  and  the  sensor  orientation. 
But  the  DOLP  on  wet  soil  appears  to  be  consistently  high, 
regardless  of  those  conditions.  This  is  illustrated  in 
Figure  13,  which  contains  a  parallel  polarization  image 
(left)  and  the  corresponding  polarization  contrast  image 
(right).  The  scene  contains  sitting  water,  wet  soil,  dry 
soil,  and  vegetation.  While  the  DOLP  of  the  sitting  water 
is  much  lower  that  the  dry  soil,  the  DOLP  of  the  wet  soil 
is  much  higher  than  the  dry  soil.  Even  mud  driven  over 
by  vehicles  has  a  higher  DOLP  than  surrounding  dry  soil 
(Figure  14). 


There  is  one  case,  however,  where  we  have  observed 
a  weak  polarization  signature  on  wet  soil.  As  illustrated 
in  Figure  15,  the  DOLP  on  wet  soil  in  shadow  is  very 
similar  to  the  DOLP  of  dry  soil  in  shadow.  In  addition, 
water  and  mud  cannot  be  distinguished  solely  using 
DOLP.  Water  can  be  distinguished  from  mud  by  using 
color  to  detect  sky  reflections  and  stereo  range  data  to 
detect  terrain  reflections  (Rankin  and  Matthies,  2006). 


Figure  13.  A  SAMBA  parallel  and  polarization 
contrast  image.  The  DOLP  on  water  can  change 
based  on  if  the  sun  is  out,  the  sun  position,  and  the 
sensor  orientation.  But  the  DOLP  on  wet  soil  appears 
to  be  consistently  high,  regardless  of  those  conditions. 


Figure  14.  A  SAMBA  parallel  and  polarization 
contrast  image.  Mud  that  has  been  driven  over  has  a 
high  DOLP. 


Figure  15.  A  SALSA  intensity  and  DOLP  image.  The 
DOLP  of  wet  soil  in  shadow  is  similar  to  the  DOLP  of 
the  dry  soil  in  shadow. 


7.  MUD  LOCALIZATION  IN  A  TERRAIN  MAP 

Mud  detected  onboard  a  UGV  should  be  placed 
within  the  world  map  the  UGV  uses  to  plan  safe  paths.  In 
this  section,  we  illustrate  the  process  with  a  multi-sensor 
based  mud  detection  solution.  A  SALSA  polarization 
camera  was  used  to  detect  mud  and  a  stereo  pair  of 
Hitachi  HV-F31  color  cameras  was  used  to  localize 
detected  mud  and  reject  false  alarms.  The  first  row  of 
Figure  16  shows  the  sensors  mounted  to  the  hood  of  a 
High  Mobility  Multi-Wheeled  Vehicle  (HMMWV).  The 
stereo  baseline  was  50cm.  All  three  sensors  were 
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calibrated  within  the  vehicle  frame.  The  HMMWV  was 
manually  driven  towards  the  mud  body  shown  in  the 
upper  left  image.  The  images  in  rows  2-4  are  from  one 
frame  of  data  during  the  drive,  illustrating  mud  detection 
from  a  range  of  27  meters. 

A  mud  detection  algorithm  was  implemented  which 
thresholds  DOLP  and  back-projects  polarization  pixels 
that  have  a  high  DOLP  into  the  left  color  image  (which  is 
registered  with  the  stereo  range  image).  The  last  row  of 
Figure  16  shows  the  mud  detection  results  in  the  stereo 
point  cloud.  As  illustrated  by  the  bird’s-eye  view  of  the 
world  map,  mud  detection  and  elevation  data  are 
accumulated  over  time  in  a  north  oriented  grid  map 
containing  40cm  resolution  cells.  Dark  blue  indicates  the 
location  of  detected  mud,  cyan  indicates  the  vehicle  path, 
white  indicates  cells  that  have  no  data,  and  the  gray  level 
represents  elevation  (where  black  is  high  elevation).  As 
we  only  expect  mud  to  occur  on  the  ground,  stereo  vision 
can  be  used  to  isolate  ground  pixels  from  the  other  pixels 
corresponding  to  ground  clutter.  For  this  particular  scene, 
vegetation  with  a  high  DOLP  was  not  placed  within  the 
world  map  since  those  pixels  are  above  the  ground.  We 
have  not  implanted  color  analysis  yet,  which  will  improve 
the  robustness  of  this  detector. 

CONCLUSIONS 

Robust  mud  detection  is  a  critical  perception 
requirement  for  UGV  autonomous  navigation.  There  are 
several  characteristics  of  mud  that  may  be  detectable  with 
appropriate  sensors.  For  example,  mud  only  occurs  on 
the  ground  surface,  mud  is  usually  cooler  than 
surrounding  soil  during  the  daytime,  mud  is  generally 
darker  than  surrounding  soil  in  visible  imagery,  and  mud 
is  highly  polarized.  None  of  these  cues,  however,  is 
definitive  on  its  own.  Dry  soil  also  occurs  on  the  ground 
surface,  shadows  are  also  cooler  than  surrounding  soil, 
shadows  are  also  darker  than  surrounding  soil  in  visible 
imagery,  and  cars,  water,  and  some  vegetation  are  also 
highly  polarized.  Thus,  a  multi-sensor  approach  to  mud 
detection  is  desired. 

A  passive  mud  detection  solution  is  desirable  to  meet 
the  Future  Combat  System  Autonomous  Navigation 
System  (FCS-ANS)  requirements.  Under  the  ARL  RCTA 
program,  JPL  has  performed  daytime  data  collections  on 
wet  and  dry  soil  using  a  variety  of  passive  sensors 
including  color  stereo,  multi- spectral  (visible  plus  NIR), 
SWIR,  MWIR,  LWIR,  and  polarization.  In  this  paper,  we 
have  described  the  cues  for  mud  detection  that  several  of 
these  passive  sensors  can  provide  under  ideal  daytime 
conditions  (isolated  wet  soil  surrounded  by  dry  soil  during 
nominal  weather),  and  a  method  for  localizing  mud 
hazards  in  a  world  map  using  stereo  vision. 

During  FY09,  we  are  planning  on  fusing  cues  from 
color  into  the  current  mud  detector,  and  implementing  a 


second  mud  detector  that  fuses  cues  from  color,  stereo, 
and  thermal  infrared.  More  work  is  needed  to 
characterize  the  usability  of  the  sensors  evaluated  in  this 
paper  to  detect  mud  under  non-ideal  conditions. 


Mud  detection  back- 

projected  in  left  color  image  Stereo  range  image 


Figure  16.  Localizing  detected  mud  in  a  world  model 
using  a  stereo  pair  of  color  cameras. 
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